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METHODS, SYSTEMS AND 
APPARATUS (NPS042) 



FIELD OF THE INVENTION 

The present invention relates to the field of mobile robotics. 



CROSS-REFERENCES 

Various methods, systems and apparatus relating to the present invention 
are disclosed in the following co-pending applications filed by the appli- 
cant or assignee of the present invention. The disclosures of all of these 
co-pending applications are incorporated herein by cross-reference. 

Filed 17 February 2002: 

Australian Provisional Application entitled "Methods, Systems and Appa- 
ratus (NPS04iy\ 

Filed 4 December 2002: 

USSN _/ entitled "Rotationally Symmetric Tags" (docket 

number NPT020US). 

Filed 22 November 2002: 

PCT/AU02/01572 and PCT/AU/02/01573. 

Filed 25 October 2002: 

Australian Provisional Application 2002952259 entitled "Methods and 
Apparatus (NPT019)". 

Filed 15 October 2002: 

PCT/AU02/01391, PCT/AU02/01392, PCT/AU02/01393, 
PCT/AU02/01394 and PCT/AU02/01395. 

Filed 26 November 200 1 : 

PCT/AU0 1/0 1527, PCT/AU0 1/0 1528, PCT/AU0 1/0 1529, 
PCT/AU01/01530 and PCT/AU01/01531. 
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Filed 11 October 2001: 
PCT/AU01/01274. 
Filed 14 August 2001: 
PCT/AU01/00996. 
Filed 27 November 2000: 

PCT/AU00/01442, PCT/AUOO/01444, PCT/AU00/0 1446, 
PCT/AU00/01445, PCT/AU00/0 1450, PCT/AU00/0 1453, 
PCT/AU00/01448, PCT/AUOO/0 1447, PCT/AU00/01459, 
PCT/AU00/0 1451, PCT/AUOO/01454, PCT/AU00/01452, 
PCT/AU00/01443, PCT/AUOO/01455, PCT/AU00/01456, 
PCT/AU00/01457, PCT/AU00/01458 and PCT/AU00/01449. 

Filed 20 October 2000: 

PCT/AU00/01273, PCT/AU00/01279, PCT/AU00/0 1 288, 
PCT/AUOO/0 1 282, PCT/AU00/0 1 276, PCT/AU00/0 1 280, 
PCT/AU00/01274, PCT/AU00/0 1289, PCT/AU00/0 1275, 
PCT/AU00/01277, PCT/AU00/01286, PCT/AU00/0 1281, 
PCT/AU00/01278, PCT/AU00/0 1287, PCT/AU00/0 1285, 
PCT/AU00/01284 and PCT/AUOO/0 1283. 

Filed 15 September 2000: 

PCT/AU00/0 11 08, PCT/AU00/0 1 1 1 0 and PCT/AU00/0 1111. 
Filed 30 June 2000: 

PCT/AU00/00762, PCT/AUOO/00763, PCT/AUO0/00761, 
PCT/AU00/00760, PCT/AU00/00759, PCT/AUOO/00758, 
PCT/AU00/00764, PCT/AUOO/00765, PCT/AUO0/OO766, 
PCT/AU00/00767, PCT/AU00/00768, PCT/AUOO/00773, 
PCT/AU00/00774, PCT/AUOO/00775, PCT/AU00/00776, 
PCT/AU00/00777, PCT/AUOO/00770, PCT/AU00/00769, 
PCT/AU00/00771, PCT/AUOO/00772, PCT/AU00/00754, 
PCT/AU00/00755, PCT/AU00/00756 and PCT/AU00/00757. 

Filed 24 May 2000: 

PCT/AU00/00518, PCT/AU00/0O5 1 9, PCT/AU00/00520, 
PCT/AU00/00521, PCT/AU00/00522, PCT/AU00/00523, 
PCT/AU00/00524, PCT/AU00/00525, PCT/AU00/00526, 
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PCT/AUOO/00527, PCT/AUOO/00528, PCT/AU00/00529, 
PCT/AUOO/00530, PCT/AUOO/00531, PCT/AUOO/00532, 
PCT/AU00/00533, PCT/AUOO/00534, PCT/AU00/00535, 
PCT/AU00/00536, PCT/AUOO/00537, PCT/AU00/00538, 
PCT/AUOO/00539, PCT/AU00/0054O, PCT/AU00/00541, 
PCT/AU00/OO542, PCT/AU00/OO543, PCT/AU00/00544, 
PCT/AUOO/00545, PCT/AU00/00547, PCT/AU00/00546, 
PCT/AUOO/00554, PCT/AU00/00556, PCT/AU00/00557, 
PCT/AU00/00558, PCT/AUOO/00559, PCT/AU00/00560, 
PCT/AU00/00561, PCT/AUOO/00562, PCT/AUOO/00563, 
PCT/AU00/0O564, PCT/AUOO/00565, PCT/AUOO/00566, 
PCT/AUOO/00567, PCT/AU00/00568, PCT/AU00/00569, 
PCT/AU0O/OO570, PCT/AU00/0O57 1 , PCT/AU00/00572, 
PCT/AUOO/00573, PCT/AUOO/00574, PCT/AUOO/00575, 
PCT/AU00/00576, PCT/AU00/00577, PCT/AUO0/00S78, 
PCT/AU00/00579, PCT/AU00/0058 1 , PCT/AUOO/00580, 
PCT/AUOO/00582, PCT/AU00/00587, PCT/AUOO/00588, 
PCT/AU00/00589, PCT/AU00/00583, PCT/AU00/00593, 
PCT/AU00/00590, PCT/AU00/00591, PCT/AU00/00592, 
PCT/AU00/0O594, PCT/AU00/00595, PCT/AUOO/00596, 
PCT/AU00/00597, PCT/AU00/00598, PCT/AUOO/00516, 
PCT/AU00/005 1 7 and PCT/AU00/OO5 1 1 . 



BACKGROUND OF THE INVENTION 

Principles of mobile robotics are described in detail in Dudek, G, and M. 
Jenkin, Computational Principles of Mobile Robotics (Cambridge Univer- 
sity Press, 2000) and Nehmzow, U., Mobile Robotics: A Practical Intro- 
duction (Springer Verlag, 2000). Practical mobile robot construction is 
described in detail in Wise, E., Applied Robotics (Prompt Publications, 
1999) and McComb, G, The Robot Builder's Bonanza, 2nd Edition 
(McGraw Hill, 2001). 



SUMMARY OF THE INVENTION 

In a first aspect, the present invention provides a mobile robot incorporat- 
ing a sensor for sensing an identity-coding and position-coding pattern dis- 
posed on or in a surface, the sensor being configured to generate, from the 
coding pattern, a position (and an orientation) of the robot relative to the 
surface, and an identity of a region containing the position, the robot being 
configured to report the identity and the position (and the orientation) to a 
host computer, the host computer being configured to identify a robot con- 
trol program associated with the region, the robot being configured to be 
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responsive to at least one command from the robot control program to 
move to a new position (and orientation) relative to the surface. 

The robot preferably incorporates a marking device for marking the sur- 
face, and is configured to be responsive to at least one command from the 
robot control program to mark the surface using the marking device. 

The robot is preferably configured to mark the surface while moving. 

In a second aspect, the present invention provides a mobile robot incorpo- 
rating a sensor for sensing a position-coding pattern disposed on or in a 
surface, and a marking device for marking the surface, the sensor being 
configured to generate a position (and an orientation) of the robot relative 
to the surface, the robot being configured to be responsive to at least one 
command from the robot control program to move to a new position (and 
orientation) relative to the surface, and the robot being configured to be 
responsive to at least one command from the robot control program to 
mark the surface using the marking device. 

The robot is preferably configured to mark the surface while moving. 

DRAWINGS 

Figure 1 shows a plan view and two elevations of a mobile robot 230 
according to a preferred form of the present invention; 

Figure 2 shows a plan view and two elevations of the internal mechanical 
structure of the mobile robot 230 of Figure 1; 

Figure 3 shows a layout of an interactive printed chess board 250 according 
to a preferred form of the present invention; 

Figure 4 shows a block diagram of a mobile robot controller 200 and its 
associated peripherals, according to a preferred form of the present inven- 
tion; 

Figure 5 shows a communication flow between the mobile robot 230 and 
associated networked entities; and 

Figure 6 shows an application context class diagram, in UML notation, for 
a robot control application according to a preferred form of the present 
invention. 
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DETAILED DESCRIPTION 

A netpage consists of visible graphic data intended for human interpreta- 
tion, as well as substantially invisible (or at least inconspicuous) netpage 
coded data intended for machine interpretation. The netpage coded data 
identifies, at each of at least a given density of points on the netpage, the 
identity of the netpage and the two-dimensional coordinates of the point. 
The given density is typically of the order of a few millimeters. A netpage 
sensing device incorporates an optical sensor and a decoder for netpage 
coded data. When placed in a position to sense netpage coded data, the net- 
page sensing device is able to determine the identity of the netpage and its 
own position relative to the netpage from a purely local snapshot of the net- 
page. Whereas the netpage coded data only directly encodes position to a 
certain precision (e.g. of the order of a few millimeters), the netpage sens- 
ing device can determine its position and orientation relative to the netpage 
to greater precision (e.g. of the order of a few micrometers) based on the 
alignment, rotation and perspective distortion of the coded data in its field 
of view. 

Note that the distinction in a particular coding pattern between page iden- 
tity (i.e. netpage id) and point coordinates is merely a functional distinc- 
tion. An actual coding pattern may utilize any mixture (or hierarchy) of 
region identifiers and coordinates, ranging from a pure coordinate code 
where the coordinate space spans a multitude of pages, through to a pure 
identifier code where a page contains a multitude of identified regions. In 
the preferred coding, as described above, the coding pattern contains a 
multitude of identified regions, each containing a range of coordinates, and 
each conveniently corresponding to the size of a page. Note also that the 
size of a page is itself not fixed, and may correspond to the size of a sheet 
of paper (e.g. Letter/A4, Tabloid/A3, etc.), or to the size of the surface of an 
arbitrary object. 

If the scale of the netpage coding pattern is increased (e.g. so that the given 
point density is of the order of centimeters or decimeters or larger), then the 
required imaging field of view grows accordingly. However, the precision 
with which the corresponding netpage sensing device can determine its 
precision and orientation relative to the netpage is a function of the 
device's imaging resolution, not the size of its field of view. It is therefore 
possible, given sufficient resolution, to determine position and orientation 
to arbitrary precision, independent of the scale of the netpage coding pat- 
tern, subject of course to normal optical imaging constraints such as dif- 
fraction limit. 

A netpage may be printed onto a surface, such as the surface of a sheet of 
paper, using a netpage printer. Printing may take place in bulk, or on 
demand. The graphic data is typically printed using visible inks, such as 
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cyan, magenta, yellow and black inks. The coded data is typically printed 
using an invisible ink such as an infrared-absorptive ink, or using a 
low-visibility ink. 

More generally, the graphic data may be printed or otherwise deposited on 
or in the surface by any suitable device or process, and the coded data may 
be printed or otherwise deposited on or in the surface by any suitable. The 
two devices and/or processes may be entirely unrelated, and need not oper- 
ate simultaneously. It is within the scope of the present invention that the 
pattern of the coded data is represented in any way that allows it to be 
sensed, e.g. optically, magnetically, chemically, etc. 

A netpage disposed on a surface is backed by a description of that netpage 
stored in a computer system, indexed by the netpage id. When a netpage 
sensing device interacts with a netpage, the sensing device forwards the 
details of the interaction to the computer system for interpretation with ref- 
erence to the stored netpage description. The forwarded details typically 
include the decoded netpage id and the decoded position of the sensing 
device relative to the netpage. The details may also consist of a stream of 
netpage ids and positions. When the netpage sensing device is in the form 
of a writing implement or stylus, and the stream is therefore representative 
of motion of the writing implement or stylus relative to the netpage, the 
stream is referred to as digital ink. The netpage sensing device then typi- 
cally incorporates a contact sensor, and is configured to begin generating 
the stream when it comes into contact with the surface, and cease generat- 
ing the stream when contact with the surface is broken. Each digital ink 
stream delimited by a contact event and a loss of contact event is referred 
to as a stroke. The computer system retrieves the netpage description corre- 
sponding to the netpage id embedded in the stroke, and interprets the stroke 
with respect to the semantics of the netpage description. For example, if the 
netpage description describes a text field with a specific position and 
extent, the computer system may determine whether the stroke intersects 
the text field, and if so may interpret the stroke, in conjunction with other 
strokes similarly assigned to the text field, as handwritten text, and attempt 
to convert the strokes to a string of identified characters. If the netpage 
description describes an action zone (also referred to as a hyperlink) with a 
specific position and extent, the computer system may determine whether 
the stroke intersects the zone, and if so may interpret the stroke as invoking 
the action, which may in turn cause the computer system to send a corre- 
sponding message to another application associated with the action. Alter- 
natively, a netpage stroke is forwarded directly to an application which is 
specific to the netpage id embedded in the stroke. 

If the netpage sensing device incorporates a marking nib or printing device, 
then the computer system typically associates digital ink input from the 
device with the corresponding page by storing the digital ink in a persistent 
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manner, indexed by the netpage id. In this way the digital ink can be repro- 
duced when the page is re-printed, and can be searched. 

A netpage sensing device in the form of a stylus and pen is described in 
co-pending PCT application WO 00/72230 entitled "Sensing Device, filed 
24 May 2000; and co-pending US application USSN 09/721,893 entitled 
"Sensing Device", filed 25 November 2000. A netpage sensing device in 
the form of a viewer is described in co-pending PCT application WO 
01/41046 entitled "Viewer with Code Sensor", filed 27 November 2000. 

Various netpage coding schemes and patterns are described in co-pending 
PCT application WO 00/72249 entitled "Identity-Coded Surface with Ref- 
erence Points", filed 24 May 2000; co-pending PCT application WO 
02/84473 entitled "Cyclic Position Codes", filed 11 October 2001; 

co-pending US application USSN _/ entitled "Rotationally 

Symmetric Tags", (docket number NPT020US) filed 4 December 2002; 
and Australian Provisional Application 2002952259 entitled "Methods and 
Apparatus (NFT019)", filed 25 October 2002. 

ROBOT WITH CODE SENSOR 

A mobile robot is typically an electro-mechanical device incorporating a 
controller and a steerable propulsion system. It may incorporate sensors for 
reacting to its environment. For example, it may incorporate one or more 
contact sensors so that when it bumps into an obstacle it can take remedial 
action such as reversing and proceeding in a different direction. A more 
sophisticated mobile robot may have access to a map of its environment, 
and may navigate from one point to another with reference to the map. It 
may also create a map of its environment during its explorations, either via 
its contact sensors, or via a more sophisticated sensory or vision system. 
Principles of mobile robotics are described in detail in Dudek, G, and M. 
Jenkin, Computational Principles of Mobile Robotics (Cambridge Univer- 
sity Press, 2000) and Nehmzow, U., Mobile Robotics: A Practical Intro- 
duction (Springer Verlag, 2000), the contents of both of which are 
incorporated herein by cross-reference. Practical mobile robot construction 
is described in detail in Wise, E., Applied Robotics (Prompt Publications, 
1999), and McComb, G, The Robot Builders Bonanza (Second Edition, 
McGraw Hill, 2001), the contents of both which are incorporated herein by 
cross-reference. 

The netpage robot 230, a preferred form of which is illustrated in Figure 1 
and Figure 2, is a mobile robot incorporating a differential drive system 
steered by an on-board controller 200. The drive system includes a pair of 
DC motors 206, with each motor coupled to a wheel 208 via a gearbox 207. 
Each motor is controlled by a motor controller 205, such as a discrete or 
integrated H-bridge to provide direction and braking control, and a 
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pulse-width modulator to provide speed control. The drive system may 
alternatively utilize a pair of stepper motors 206, in which case each motor 
controller 205 generates the appropriate pulse sequences to control the 
speed and direction of its corresponding stepper motor. A castor 209 pro* 
vides a third point of support for the robot. The robot incorporates a plat- 
form 211 to which the motors and gearboxes are attached, and which is 
itself attached to the robot housing 210 (shown enclosing the robot in Fig- 
ure 1 and in outline in Figure 2). The platform 21 1 also supports the main 
PCB 212 which holds the controller 200 and other electronic components. 

The netpage robot has a wireless communications link to a host computer 
system. It utilizes a radio transceiver 189 (e.g. operating according to a 
short-range standard such as Bluetooth, or a longer-range standard such as 
GSM, GPRS or other CDMA variant). Alternatively it may utilize an infra- 
red transceiver (e.g. operating according to the IrDA standard). It is pow- 
ered by a rechargeable battery 139. 

When placed on a netpage, the netpage robot 230 determines, via its 
in-built netpage sensor, the id 50 of the netpage and its own position (and 
orientation) 221 relative to the netpage. It then reports this information to 
the host computer via the communications link. The netpage sensor incor- 
porates infrared illumination LED(s) 131, an area image sensor 132, and 
optics 135 including an infrared filter, aperture and focussing optics. The 
robot detects contact with a surface via contact switch 194, or alternatively 
by successfully sensing and decoding the surface coding pattern. 

The robot has a unique id 220 which it communicates to the host and which 
identifies it uniquely among netpage robots. The robot is responsive to a 
command 222 from its host to move to a specified position on a specified 
netpage. When the robot reaches its target position it reports its arrival to 
the host. Because the robot receives continuous position information from 
its netpage sensor, it is able to navigate with absolute accuracy, without the 
cumulative errors which plague odometry systems (due to such things as 
wheel slip and inaccurate kinematic modelling), and therefore without the 
need to include more sophisticated position-correction mechanisms. 

To allow the robot to move with maximum efficiency, a movement com- 
mand is usefully specified in terms of a polyline or piece-wise curve. This 
allows the robot to follow a multi-segment path without slowing to a stop 
between segments. The robot may also be responsive to overlapped com- 
mands, where it is able to receive and buffer one or more new commands 
before completing a current command. In the case of a polyline, the robot 
may optionally be instructed to interpolate a smooth curved path through 
the vertices of the polyline. 
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Commands to the robot may specify speed, or the robot may be allowed to 
choose its own speed according to its capabilities and the complexity of the 
path. If the speed is specified, then the polyline (or piece-wise curve) may 
contain a number of speed specifications (e.g. at each vertex), which are 
interpolated by the robot. 

The size of the netpage robot may vary from small (e.g. millimeter or cen- 
timeter scale) to large (e.g. decimeter or meter scale). The netpage coded 
data pattern may be scaled to match the scale and intended applications of 
the robot. However, a single coded data pattern scale may be used across 
the full range of robot scales. 

PAGE-DRIVEN ROBOT BEHAVIOR 

For the purposes of the following discussion, the game of chess is used for 
illustrative purposes. Note, however, that the principles are general and 
apply to games other than chess, and to applications beyond games. 

The netpage chess board 250, as shown in Figure 3, may be pre-made or 
may be printed on demand using a netpage printer. The board incorporates 
a playing area 254, consisting of an 8x8 checkerboard pattern, as well as 
additional areas and controls. When a netpage robot is placed on the net- 
page chess board, it initially reports its own position and orientation and 
the netpage id 50 of the netpage chess board to the host computer. The page 
server 841 on the host computer identifies the chess application associated 
with the netpage id of the netpage chess board (or associated with a zone of 
a field embedded in the netpage description associated with the netpage id), 
and forwards all robot input to the chess application. The host computer 
also forwards robot control commands from the chess application to the 
robot. The chess application may be local to the host computer, or may be 
executing on a different computer on a local-area or wide-area network 
(e.g. the Internet). 

Since chess involves sixteen pieces, the chess application receives sixteen 
initial position reports from sixteen participating robots. If it receives less 
than sixteen initial position reports then it can signal an error to the user, 
e.g. via an audible or printed output. If it receives more than sixteen initial 
position reports, then it can instruct the redundant robots to move off the 
chess board. The chess application assigns each robot the role of one of the 
sixteen chess pieces, and instructs each robot to move to its starting posi- 
tion. The chess application must plan each robot's path to avoid collisions 
with other robots. For chess, a simple heuristic consists of the following: 

• assign each back-line (i.e. non-pawn) role to the nearest robot 

♦ assign each front-line (i.e. pawn) role to the nearest remaining robot 
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• move to its starting position any robot not yet in its starting position 
which has an un-obscured straight-line view of its starting position 

• repeat until all pieces are properly positioned or no more pieces can be 
moved 

• move any remaining pieces using two-part moves 

Clearly more sophisticated algorithms are possible, many of which result in 
more rapid positioning. 

The user makes a move by picking up a robot and placing it at a new posi- 
tion. The robot reports its new position to the chess application. If the move 
is the first move of a game then the "color" of the robot determines whether 
the user has chosen to play as white or black. Each robot ideally incorpo- 
rates some kind of display 186 to indicate which role it has been assigned 
(e.g. white queen or black king). The display may be in the form of a small 
alphanumeric LCD showing an alphanumeric representation of the role, or 
a monochrome or color bit-mapped LCD or OLED showing an icon of the 
role. The robot is responsive to a command from the host (i.e. from the 
chess application via the host) to program the content of the robot's display. 
The robot may have multiple displays. For example it may have one dis- 
play on its top and another on its face. Its entire visible surface may be 
completely covered by one or more displays, allowing a semi-three-dimen- 
sional icon to be displayed on the robot. 

The chess application may chose to position the moved robot more cen- 
trally in its square, to allow more uniform navigation of other robots. It 
may also normalize its orientation, so that it is always facing the opposi- 
tion. If the move is illegal, or if the user mistakenly moves an opposing 
piece, then the chess application can issue a spoken warning (e.g. via the 
robot's speaker 204, display 186, or status LED(s) 116) and reverse the 
move. If a piece is accidentally placed off the board, then the chess applica- 
tion can once again issue a warning. 

The chess application then computes its responding move, and instructs 
one or more of its robots to move accordingly. Chess-playing algorithms 
and software are described in detail in Pawlak, R.J., Chess Software Sour- 
cebook (Treehaus Books, 1999) and Heinz, E.A., Scalable Search in Com- 
puter Chess: Algorithmic Enhancements and Experiments at High Search 
Depths (Morgan Kaufrnann, 1999), the contents of both of which are incor- 
porated herein by cross-reference. 

The user can also play against a remote human opponent, in which case 
each local move by the user is tracked by the chess application, and is mir- 
rored, under chess application control, by the corresponding remote piece. 
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In the case of pieces which are allowed to move despite being hemmed in, 
such as knights, or pieces which are allowed to move diagonally, such as 
bishops and queens, the chess application may temporarily move offending 
robots out of the way, irrespective of their color. It may also temporarily 
move robots off the playing area 254, e.g. during castling. The chess appli- 
cation maintains a map of the chess board for navigation purposes. This 
may be the same map it uses for game-playing purposes, or separate from 
it. When the user captures an opposing piece s/he simply places the piece 
off the playing area. When the chess application captures an opposing 
piece (i.e. one of the user's pieces), it instructs the piece to move off the 
playing area. The chess board includes an area 251 for out-of-play pieces 
of each color. The chess application can automatically move a piece cap- 
tured by the user to the out-of-play area, irrespective of where on the board 
the user places the piece after capture. If the chess board is upset and some 
pieces lose their places, then the chess application can automatically return 
each piece to its proper place. Alternatively the chess board can include a 
<resume> zone (not shown) which, when activated by the user with a piece 
(or other netpage sensing device), can instruct the chess application to 
return each piece to its proper place. To allow the user to resign, the chess 
board includes a <resign> zone 253 which, when activated by the user with 
a piece (or other netpage sensing device), or specifically with the resigning 
king, signals the chess application that the user is resigning. The chess 
application signals that it is resigning by moving its king off the playing 
area, or specifically to its own <resign> zone. The board also includes a 
<new game> zone 252 which, when activated by the user with a piece (or 
other netpage sensing device), signals the chess application to abandon the 
current game and set up a new game. There may be multiple <new game> 
zones corresponding to different levels of game-play by the chess applica- 
tion, e.g. ranging from beginner through intermediate to advanced. 

When a pawn reaches the opposing back line and is transformed into a 
queen, the chess application re-assigns the role of the robot, and updates 
the robot ! s display(s) accordingly. 

The netpage robot bay holds a number of netpage robots. The surface of the 
bay is netpage position-coded to allow robots to navigate into and within 
the bay under host control. The bay incorporates a netpage sensor which 
allows it to detect when and where it is placed on a netpage, and a commu- 
nications link to the host computer which allows it to report the id of the 
netpage it is placed on and its own position and orientation relative to the 
netpage. When the bay is placed on a netpage on which one or more net- 
page robots are located, the host computer can automatically instruct the 
robots to enter the bay. To this end the bay has a gentle ramp at its entry 
which a netpage robot is able to navigate. The chess board (or bay, or both) 
may also include a <gather> zone which, when activated by the user with a 
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piece (or other netpage sensing device), signals the host to gather the robots 
in the bay. 

The netpage robot bay incorporates a battery charging circuit for re-charg- 
ing the batteries of the robots in the bay. The bay has an external power 
connection, and may incorporate a larger storage battery for offline opera- 
tion. A robot may be automatically re-charged in the bay, or may re-charge 
in response to a re-charge instruction from the host. 

The netpage robot may additionally incorporate a directional contact sen- 
sor, allowing it to detect and report to the host the presence of any unex- 
pected obstacles. Such obstacles are "netpage-invisible" if they do not 
themselves incorporate netpage sensors or do not report their presence (and 
position) to the host. Any netpage robot is by definition netpage- visible and 
can therefore not become an unexpected obstruction. 

To support games such as checkers which allow pieces to be stacked, the 
netpage robot is designed to be stacked. To this end its top surface is sur- 
rounded by a rim which is designed to mate with the undercarriage of 
another robot (this detail is not included in Figure 1 and Figure 2). 

To support games such as checkers and go which only involve two kinds of 
pieces, the netpage robot may incorporate a simpler display or indicator 
system (such as an LED or a two-color LED), which can indicate which of 
the two roles the robot has been assigned. 

There are many alternatives to the architecture described above, i.e. where 
a host computer mediates communication between robots and a (possibly 
remote) chess application. For example, one of the robots may be automat- 
ically designated as the master robot for the duration of the game (or 
longer), and may download the chess application for execution on its 
embedded processor. In this case the master robot controls the slave robots. 
As another example, the robots may act as peers, and may each download 
the chess application (or parts of it) and execute it collaboratively, commu- 
nicating directly among themselves. 

The key feature of the netpage robot system architecture is that the behav- 
ior of a collection of one or more robots derives from the particular netpage 
on which they are placed. When placed on a chess board the robots behave 
as chess pieces; when placed on a checkers board they behave as checkers 
pieces; etc. This is enabled by the netpage id (either directly or indirectly) 
identifying the application, the application controlling the robots (either 
remotely or embedded), and the robots navigating absolutely using the net- 
page position coding. 
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MARKING ROBOT 

The netpage robot may incorporate a marking device for marking a surface 
such as a paper surface. The marking device is under robot control, so that 
the robot can turn marking on and off in a position-dependent way. This 
may be effected using a digitally-controlled marking device 195 such as a 
Memjet™ printhead, or with a more traditional pen which can be moved 
into contact with the surface under robot control. The robot can support 
multiple colors via a multi-color printhead, or via multiple printheads, or 
via multiple colored pens. Suitable linear printheads are described in 
co-pending PCT applications WO 01/89839 entitled "Ink Jet Printhead 
Having a Moving Nozzle with an Externally Arranged Actuator", filed 19 
October 2001, WO 01/89844 entitled "Ink Jet Printhead Nozzle Array", 
filed 24 May 2000, and related applications incorporated by cross-refer- 
ence therein and herein. Suitable circular printheads are described in 
co-pending PCT application WO 02/34531 entitled "Printhead for Pen", 
filed 19 October 2001. 

Multiple robots can also be used, each carrying a differently-colored pen. 
These can be controlled sequentially or simultaneously to draw different 
parts of a multi-colored drawing. 

When the user inserts a new pen cartridge into a robot, the robot may auto- 
matically determine the color of the pen by decoding a barcode on the sur- 
face of the cartridge, or by reading a value from a solid-state memory 
embedded in the cartridge. Alternatively, the user may activate a 
color-coded netpage hyperlink using the robot (or other netpage sensing 
device), thus notififying the computer system and/or corresponding appli- 
cation of the robot's new pen color. A palette of such color-coded hyper- 
links may be provided on a separate general-purpose netpage or may be 
incorporated into the design of an application-specific netpage. 

The marking robot may incorporate one or more printheads with a signifi- 
cant width, e.g. of the order of the diameter of the robot itself (as shown in 
Figure 2). This allows the robot to draw lines with subtle structure, such as 
varying width and varying color texture. It also allows the robot to operate 
in raster mode, as opposed to (or in addition to) the vector mode described 
so far. In raster mode the robot can transfer a two-dimensional image to a 
surface in a single sweep, or build up a larger image in multiple sweeps. 
The source image may be a bit-mapped image such as a photograph, or it 
may be a set of vector strokes, e.g. corresponding to handwritten text. 
Reproducing handwritten text in raster mode may be significantly faster 
than reproducing it in vector mode, particularly if the height of the text is 
less than the Width of the printhead(s). In raster mode the computer system 
(or the robot) keeps track of exactly which parts of the source image have 
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been transferred to the surface, allowing the robot to transfer the image to 
the surface without gaps or overlaps between successive sweeps. 

The marking robot, in addition to being responsive a command from its 
host to move to a specified position on a specified netpage, is also respon- 
sive to a command from its host to draw a line of a specified color while 
moving to the target position. As before, the host mediates communication 
between the robot and an application which is specific to the netpage (or 
netpage region) on which the marking robot is placed. The host may hide 
the drawing functions of the robot behind a line-drawing interface which it 
presents to the application. The application is then unaware of how the 
drawing functions are implemented. The robot may directly support the 
drawing of curved lines, i.e. during movement along curved paths, or the 
application (or host) may approximate curved lines by instructing the robot 
to draw a succession of shorter straight-line segments. 

The line-drawing interface, or the robot itself, may be responsive to Post- 
Script drawing commands (refer to Adobe Systems Incorporated, Post- 
Script Language Reference (3rd Edition, Addision- Wesley 1999), the 
contents of which are incorporated herein by cross-reference). 

When the marking robot marks a netpage, the markings persist as part of 
the netpage in the usual way. In this way they become persistent, reproduc- 
ible, and searchable. 

Applications of the marking robot include (but are not limited to): pure 
plotting of drawings, particularly where it is impractical for the user to uti- 
lize a large-format printer, but it is practical for the user to obtain a 
large-format pre-netpage-coded "blank"; collaborative markup, where each 
participant's contribution, typically hand-drawn or handwritten with a net- 
page pen, is robotically mirrored on each of the other participants' 
"work-in-progress" netpages; and system feedback in the absence of a 
page-oriented printer. 

REMOTE CONFERENCING AND COLLABORATIVE MARKUP 

As described in the context of the chess application, when the user places 
one or more robots on a collaborative markup netpage the corresponding 
collaborative markup application can automatically assign each of the 
robots to represent one of the remote collaborative participants. If names, 
icons and/or photos representing the participants* are available, then this 
information can be displayed on the robots. In the limit case, each robot's 
display can act as a video conferencing display for the remote participant it 
represents, i.e. continuously displaying video of the remote participant cap- 
tured remotely via a video camera and transmitted over the network. The 
robot may also usefully incorporate a speaker 204 for relaying the remote 
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participant's voice, and a microphone 201 for capturing voice input. The 
microphone may also be provided separately, e.g. attached to the host com- 
puter (or to the relay - see below). 

A remote conferencing netpage application is described in co-pending PCT 
application WO 01/02940 entitled "Method and System for Conferencing", 
filed 30 June 2000. A collaborative document markup netpage application 
is described in co-pending PCT application WO 01/02939 entitled 
"Method and System for Collaborative Document Markup", fited 30 June 
2000. 

MIRRORING ROBOT MOVEMENT 

In the same way that a netpage robot can mirror the movement of a remote 
user's netpage pen, a robot can mirror the movement of a remote robot. The 
remote robot may be under direct or indirect user control, i.e. it may be 
being moved by the user's own hand, or it may be being moved under 
interactive computer control. For example, in an interactive tank game 
being played by multiple remote participants, each user may control their 
local tank via a graphical user interface on a PC. The tank may include a 
video camera for capturing a tank Veye view of the game environment, and 
the video can be shown on the user's PC display together with various tank 
navigation controls. Alternatively the video can be shown on a conven- 
tional television set receiving the video wirelessly from the tank, independ- 
ently of the computer system. In this case the user can also control the tank 
via a printed netpage tank control user interface and a netpage stylus, i.e. 
without any direct interaction between the user and a PC. 

Each user may have an additional local tank robot representing each 
remote user's tank. Each such tank simply mirrors the corresponding 
remote tank's movements, under the control of the network-resident tank 
game application. 

The robots may have a game-specific shape (such as the shape of a tank), 
or may be generic. They may also be general-purpose robots which can be 
inserted into low-cost game-specific bodies (e.g. tanks) for the purposes of 
specific games. 

MOTION RECORDING AND PLAYBACK 

As mentioned above, a user may directly move a robot, i.e. by hand, and 
have the movement mirrored by a remote robot. The user may also instruct 
a robot to record a hand-specified movement, and then have the robot 
replay the recorded movement. Recording (and playback) may be initiated 
by depressing a button on the robot, or by providing a control input through 
some other interface, such as by designating a <record> zone (or <play- 
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back> zone) on a netpage with a netpage sensing device (including the 
robot itself), or by specifying a command by other means, such as key- 
board or mouse input to a computer system. 

Motion recording and playback may be provided as a generic function of 
the host computer system, or may be provided by specific applications. In 
either case recording and playback rely on already-described mechanisms. 
Hand-specified motion may optionally be subject to smoothing by the 
computer system before playback, and a robot may also be instructed to 
follow a path at a different speed to the varying speed associated with the 
original hand-specified motion. 

ROBOTIC VIEWER 

A netpage viewer is a device which, when placed on a netpage, provides a 
registered interactive view of elements of the page underlying the viewer. 
The viewer incorporates a netpage sensor for determining its own position 
and orientation relative to the netpage, and to allow it to obtain interactive 
content specific to the page from the netpage system. A netpage viewer is 
described in co-pending PCT application WO 01/41046 entitled "Viewer 
with Code Sensor", filed 27 November 2000. A netpage viewer incorporat- 
ing a printer is described in co-pending PCT application WO 01/41045 
entitled "Viewer with Code Sensor and Printer", filed 27 November 2000. 

A netpage viewer is usefully augmented with robotic capabilities, to allow 
the viewer to mirror movement of a remote viewer, or to allow the viewer 
to display remote netpage pen input on its display, wherever the input may 
appear on the page. The robotic netpage viewer will move to the area of the 
page where input is occurring, to allow the input to be displayed in regis- 
tered fashion. This is particularly useful in a collaborative markup context 
as described above. The user may at any time move the viwer to a different 
part of the page to view earlier markup. If the viewer incorporates a printer, 
then remote (and local) markup can be simultaneously (and persistently) 
transferred to the underlying physical page. 

ROBOT CONTROLLER 

Figure 4 shows a block diagram of the robot controller 200 and its associ- 
ated peripherals. The integrated controller chip incorporates a processor 
145 connected to a number of other components via a shared bus 146, 
including a working memory 148 (such as an SRAM). An off-chip 
non-volatile memory 147 (such as a flash memory) holds the fixed robot 
program and other persistent data. The controller includes a parallel inter- 
face for controlling the motor controllers 205, status LED(s) 116, and illu- 
mination LED(s) 131, and for sensing the optional contact switch 194. It 
includes a serial interface for communicating with a host computer (and/or 
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other robots) via the radio transceiver 189, and for controlling the optional 
printhead 195. It includes an image sensor interface 152 for the netpage 
sensor's image sensor 132, an optional display interface 185 for the display 
186, an optional ADC 202 for the microphone 201, and an optional DAC 
203 for the speaker 204. 

ROBOT COMMUNICATION FLOW 

Figure 5 shows a possible communication flow between a netpage robot 
230 and associated networked entities in the context of the netpage system. 

When the robot is placed on a netpage, its goal is to notify its presence to a 
corresponding robot control application 804, and begin receiving com- 
mands from the application. The robot typically has short-range wireless 
connectivity with a relay device 44, which itself typically has longer-range 
network connectivity to netpage system servers via wired LAN, WAN or a 
longer-range wireless link. The relay device may be a mobile phone, as 
described in co-pending PCT application WO 00/72286 entitled "Relay 
Device", filed 24 May 2000; an interactive netpage printer, as described in 
co-pending PCT application WO 02/42894 entitled "Interactive Printer", 
filed 26 November 2001; etc. 

The robot transmits its initial position report to the relay, including its robot 
id 220, the netpage id 50 of the page on which it finds itself, and its posi- 
tion (and orientation) 221 on the page. The relay is able to determine, with 
reference to one or more id servers 12, the server id 53 of the page server 
84 1 which "owns" the page description of the page. Alternatively, the robot 
may itself be able to determine the server id 53 from the id servers) 12, 
transparently via the relay 44. The robot may also implicitly incorporate 
the relay function if it has its own longer-range network connectivity. 

The relay (or robot) then forwards the position report to the page server 
841. The page server in turn determines which application 804 to forward 
the position report to. In the general netpage scheme, this consists of 
hit-testing the robot position against the various interactive elements 
embedded in the description of the page. For the purposes of robot control, 
all robot movement is usefully forwarded directly to the application with- 
out further interpretation by the page server. It is also possible to dispense 
with the page server entirely, and forward a robot position report directly 
from the robot (or relay) to an application as identified by the id servers), 
or even as identified directly by the page id 50 embedded in the original 
page. Note that while the ids used in the netpage system are typically 
described as "pure" ids which do not support routing per se, it is also possi- 
ble, for example, for these ids to be partial or complete Internet Protocol 
(IP) addresses, either in 32-bit (IPv4) or 128-bit (IPv6) form, thus directly- 
supporting routing. 



Confidential 



S Overt rook Research 



Methods, Systems and Apparatus (NPS042) 



18 



The application 804 eventually receives the robot position report by one of 
these mechanisms, and then interprets the position report with reference to 
the layout and semantics of the page identified by the page id 50. Since the 
application may be specific to a particular page layout (e.g. to die chess 
board), it may be able to assume the layout and simply use the page id to 
index a particular instance (e.g. a particular in-progress chess game). The 
application is then able to send commands 222 directly (or indirectly) to 
the robot, and receive further position input from the robot 

As mentioned earlier, there are many possible variations on this architec- 
ture, including downloading application logic into one or more robots, and 
distributed peer-to-peer robot control. 

APPLICATION CONTEXT CLASS DIAGRAM 

Figure 6 shows a generic application context class diagram, in UML nota- 
tion, for a robot control application. The overall context 260 (e.g. corre- 
sponding to the chess application) includes a number of individual 
instances 261 (e.g. corresponding to individual chess games), each of 
which includes a number of robot roles 262 (e.g. corresponding to individ- 
ual chess pieces' roles). The context also includes a number of known 
robots 263, each of which may optionally have been assigned a role 262. 
Each instance 261 is indentified by an instance id 265. The instance id may 
correspond to the netpage id of the corresponding page, or may correspond 
to a transaction id, embedded in the page description of the corresponding 
page, which gets forwarded to the application during robot interactions. 
Each role 262 is identified by a role id 266, and has a status 267 (e.g. indi- 
cating whether a chess piece is still in play or has been captured). Each 
robot 263 is identified by a robot id 220, i.e. the robot id of the correspond- 
ing physical robot 230, and a current position 269 derived from the most 
recent position 22 1 reported by the robot. 

The present invention has been described with reference to a preferred 
embodiment and number of specific alternative embodiments. However, it 
will be appreciated by those skilled in the relevant fields that a number of 
other embodiments, differing from those specifically described, will also 
fall within the spirit and scope of the present invention. Accordingly, it will 
be understood that the invention is not intended to be limited to the specific 
embodiments described in the present specification, including documents 
incorporated by cross-reference as appropriate. 
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